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Headline findings 



This study found that the NERF Bulletin is well positioned among other kinds of 
products available for communicating research to teachers and lecturers. 

The Bulletin offers quality-assured research summaries with potential appeal 
to teachers and support staff. It also provides sources of further information. 
These features make it of potential interest to teachers, who see it as a useful 
addition to the market place. 

Teachers will tend to skim-read the Bulletin, only choosing to read items of par- 
ticular relevance to their work. For this reason, design features that make the 
content easy to navigate are much appreciated. 

The current appearance of the Bulletin is not sufficiently appealing and would 
deter teachers from wishing to engage with it. Suggested improvements include 
better layout, more space on the page, use of colour and visual elements. 

Teachers are likely to use the Bulletin in a number of ways, including: 

• for general information 

• to follow up items of particular interest with further reading 

• to recommend items to colleagues 

• to contribute to further study 

• to inform their own research 

• to validate their practice 

• to change their practice. 



1 Introduction 



The Department for Education and Skills (DfES) on behalf of the National 
Education Research Forum (NERF) has commissioned two issues of an evi- 
dence-based bulletin for practitioners. The Bulletin aims to provide a conduit 
for research to reach teaching communities across England. In doing so it is 
hoped that the Bulletin will make an important contribution to school and col- 
lege improvement and teachers’ professional development. The NFER has been 
commissioned by NERF to evaluate the effectiveness of the Bulletin by obtain- 
ing a representative cross-section of views from teachers and local education 
authority (LEA) staff. 

In phase one of the evaluation, the research team carried out a questionnaire 
survey of teachers in English nursery, primary, secondary and special schools. 
The survey sought to identify both the extent to which teachers currently read, 
apply or carry out research and also their views on the potential effectiveness of 
the Bulletin (Taggart et al., 2004). The current report will therefore concentrate 
on findings derived from phase two of the evaluation. It had the following aims: 

• to help position the NERF Bulletin among other kinds of products available 
for communicating research to teachers 

• to provide information about the way teachers might use the NERF Bulletin in 
developing their practice and in school improvement. 

To meet these aims, the research involved seven focus group interviews with 
school and college staff (40 people in all) and telephone interviews with four 
people responsible for continuing professional development in LEAs. 

1.1 About this report 

The following section sets a context for the evaluation and summarises key find- 
ings from research into teachers’ use of research. Chapter 3 contains some brief 
details about this evaluation and chapter 4 considers practitioners’ views on 
appraising and accessing research. Chapter 5 describes practitioners’ views on the 
Bulletin and chapter 6 discusses the Bulletin in relation to other publications and 
websites. The report ends with conclusions and recommendations (chapter 7) and 
an appendix containing further information about the study’s sample and methods. 



2 Teachers’ engagement with 
research: a context 



In a keynote address to the Teacher Training Agency Annual Conference in 1996, 
Professor David Hargreaves made the case for teaching to become an evidence- 
based profession (Hargreaves, 1996). He presented a challenge to teachers to 
develop a stronger emphasis on research-informed practice and for researchers to 
disseminate research in more accessible ways. This has generated much debate, 
both in research and policy communities, on what these accessible ways might 
be . An important element of this discussion has been to draw on research into 
practitioners’ use of research information. 

2.1 Influences on teachers’ use of research 

A recent NFER research project considered the role of LEAs in promoting the 
use of research for school improvement (Wilson et ah, 2003). The project began 
with a comprehensive review of the literature on the use of research to improve 
professional practice (Hemsley-Brown and Sharp, 2003). Further work at the 
NFER and elsewhere (Rickinson et ah, 2003; NCSL, 2003; Walter et ah, 2004; 
NERF, 2001) has added to our understanding of the key issues and has enabled us 
to identify the main barriers to and facilitators of practitioners’ use of research. 
The main barriers to their use of research evidence concern: 

• time 

• access 

• relevance 

• the influence of teachers’ professional culture. 

The main factors facilitating practitioners’ use of research evidence are: 

• making research-based information readily available 

• ensuring that research addresses practitioners’ interests 

• ensuring that the presentation of research information is clear, concise and 
appealing to the audience 



• reassuring practitioners that the information is from a trustworthy source and 
is well conducted 

• making explicit the generalisability and practical application of the research 

• providing evidence of the benefits of using research, especially from fellow 
practitioners 

• providing incentives and rewards for using research. 

In addition to these specific points about teachers’ use of research evidence, it is 
important to consider the relationship between engagement with research and 
engagement in research. Both may be seen as contributing to the development of 
improving pedagogy and reflective practice, which many have argued to be at the 
heart of professionalism. As McBeath et al. (1996) argue: ‘The commitment to crit- 
ical and systematic reflection on practice as a basis for individual and collective 
development is at the heart of what it means to be a professional teacher’ (p. 7). 
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3 About this study 



Focus group interviews were conducted in seven institutions across four educa- 
tional phases - nursery, primary, secondary and further education (FE). Each 
focus group comprised a broad range of practitioners, including headteachers, 
teachers/lecturers, nursery nurses and teaching assistants. The size of the groups 
ranged from five to eight individuals (see Appendix for further information). 

The NFER approached institutions that were already known to be research- 
engaged. For a more in-depth discussion of the research-engaged school see 
McBeath et al. (1996). Those who agreed to take part were interested in finding 
out more about research in general and the Bulletin in particular. For this reason, 
their views cannot be considered to be typical of practitioners across England. 
On the other hand, by approaching those with an interest in research, it is hoped 
that this evaluation will provide an indication of the potential for the Bulletin to 
contribute to evidence-based practice. 

The main aim of the focus group interviews was to find out about the potential of 
research information to inform practice. More specifically we asked practitioners 
about: 

• the types of research they relate to 

• the methods they use to find out about research 

• their views on the second edition of the Bulletin in relation to other research- 
based publications and websites. 



4 Practitioners’ views on appraising 
and accessing research 



The focus group participants were asked about the kind of research they consid- 
ered to be valuable; the kind of research ‘they related to’. There was a high 
degree of consensus across all focus groups in addressing this question. Research 
seen as valuable was considered to possess the following qualities: 

• relevance to the audience 

• concise/succinct presentation 

• easy to skim and well signposted for the reader 

• inclusion of case studies 

• easy to translate into classroom practice. 

These findings corroborate previous research and emphasise the point that teach- 
ers will engage with research information if they consider it to be an effective use 
of their time. 

It is useful to note that there was frequent mention, among the focus group partic- 
ipants, of the work of Paul Black and Dylan Wiliam on assessment for learning 
(see Black and Wiliam, 1998; Black et al., 2002). Practitioners felt that this work 
engaged with their own reflections on classroom practice and, indeed, affirmed 
some of their intuitive perceptions concerning formative assessment. 

There was some evidence of a distinction between the expectations of experi- 
enced practitioners, who were more content to adapt research findings to their 
own ends, and those of their less-experienced colleagues, who wanted items of 
immediate practical use. At a sixth form college, for example, a newly qualified 
teacher who teaches a BTech course in construction professed his inexperience in 
addressing the gaps in key skills needed by his students: 

I am surprised I am having to do such a lot of teaching of basic communica- 
tion and maths. I don’t know how to do that best and at the moment, lam doing 
it by trial and error so I would like to know what the best practice is. Lesson 
ideas and plans would therefore be very usefid. 



4.1 Accessing research through the internet 



Focus group participants were asked how they found out about research evidence. 
The most immediate responses from all groups related to using the internet. Partici- 
pants referred to the usefulness of search engines as a starting point for finding 
research, although they commented on the difficulty of finding exactly what they 
wanted. This was well expressed in the following two quotations: the former is from 
a primary practitioner and the latter is from a secondary practitioner: 

The internet has opened up a whole new world. 1 don 7 know how you could do 

a course without the internet because at least it gives you a starting point. 

I have a preference for printed materials because it takes such a long time to 

find things on the internet. 

Participants from an FE college referred to the potential of the college intranet to 
act as a research portal. In this case, part of the intranet had been given over to an 
area for staff training and resources, where skills and expertise could be shared. 
In the other college, one of the practitioners had discovered some useful 
resources for teaching key skills on the intranet. However, she had only come 
across these by chance. She then made the point that because of the tendency for 
practitioners in the college to work in relative isolation, it was not sufficient to 
put information on the intranet without telling colleagues how to find it. 

Respondents’ reasons for the choice of intemet/intranet were the comprehensive- 
ness of information, the availability of computers and the opportunity for 
dialogue via a discussion forum. A wide range of websites were mentioned, most 
commonly those provided by the DfES and National College for School Leader- 
ship (NCSL). Others included local authority websites, TeacherNet, NFER, 
National Institute for Adult Continuing Education (NIACE) and British Educa- 
tional Communications and Technology Agency (BECTA). In one of the 
secondary schools, the special educational needs co-ordinator (SENCO) made 
used of an online forum for SENCOs, where research articles are shared. 

In addition to internet resources, DVDs were mentioned in two of the focus 
groups. These were considered to be a convenient, interactive format in which 
examples of good practice could be disseminated in a clear, graphic and straight- 
forward way which minimised the need for written text. 
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In a focus group interview held at a sixth form college, one of the practitioners 
referred very positively to Teachers’ TV as a resource for discovering new 
research-informed ideas about teaching and learning. None of the other six 
practitioners present had seen any programmes on this channel but were very 
keen to find out about it. The conversation quickly turned to details of trans- 
mission times, cost and availability, which involved everyone in the group. 



4.2 Accessing research through printed materials 

When asked about their engagement with research literature, the single most 
common publication mentioned was the Times Educational Supplement (TES). 
Although staff recognised that this was not a dedicated research publication, they 
considered that research included in the TES was likely to be relevant and clear- 
ly expressed. However, one of the primary schools did not subscribe to the TES. 
A teacher at this school explained that staff did not always want to read educa- 
tional literature: ‘sometimes you need to get away from teaching’ . 

The second most common type of literature to which focus group participants 
referred was that provided by unions and teacher associations, usually in the 
form of the union magazine. Associations for headteachers such as the National 
Association for Headteachers (NAHT) and the Secondary Heads Association 
(SHA) were mentioned most often. Teachers engaged with research through 
these publications because of existing personal subscription or membership: 
learning about research is therefore one of the perceived benefits of these publi- 
cations, along with staying abreast of news and educational events. 

The third kind of literature mentioned took the form of succinct reports and 
newsletters produced by educational bodies such as the Office for Standards in 
Education (Ofsted), the Learning and Skills Council (LSC) the NCSL ( Idr maga- 
zine) and the Institute of Education, University of London. This kind of literature, 
constituting a report or bulletin, was often issued free, received by email or down- 
loaded from a website. Research conducted by Ofsted was mentioned favourably 
in two focus groups, in particular the review of the literacy and numeracy strate- 
gies and the publication Curriculum in Successful Primary Schools. One 
practitioner made a point of reading the analysis of performance data carried out 
by the Qualifications and Curriculum Agency (QCA) every year. 
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A practitioner at the early years centre mentioned an article in the Times Educa- 
tional Supplement which she had read about two years ago. The article 
summarised research about behaviour management, focusing on reinforcement 
of positive behaviour. In particular, it showed how a teacher’s tendency to draw 
a child’s attention to something they had done well had encouraged the child to 
repeat the positive behaviour. She decided to adopt this strategy herself and was 
pleased with the results. 



As regards more specific journals and specialist publications, these were 
accessed mostly through personal subscription. Practitioners at secondary and FE 
level subscribed to specialist publications such as Teaching Geography, Pastoral 
Care, Support for Learning, and the British Journal of Psychology. Other publi- 
cations were produced by the Association for Science Education (ASE), the 
Association for Teachers of Mathematics (ATM), the National Association for 
Special Educational Needs (NASEN) and the Association for the Teaching of the 
Social Sciences (ATSS). 

Primary and nursery practitioners mentioned that, in addition to personal sub- 
scriptions, their schools had institutional subscriptions to journals such as Child 
Education and Early Education. 



In a secondary school with an established culture of engagement with research, 
staff had established a practice whereby one person would recommend a book 
( The Tipping Point by Malcolm Gladwell was mentioned) . This was then circu- 
lated, as a way of stimulating ideas, establishing a common frame of reference 
and contributing to organisational change. 



4.3 Accessing research through mediators 

An emerging theme in the focus group discussions was the role that key people 
played in acting as mediators or ‘gatekeepers’ for research evidence. Sharing of 
research was often informal, usually from headteacher/principal to other staff. 
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One secondary school had developed a more formal means of sharing research. 
It had established a research unit which worked in partnership with a local uni- 
versity to nurture research projects. The head of the research unit explained that 
part of his role was to disseminate research information throughout the school. 



A year 1 teacher and primary strategy coordinator endorsed the importance of 
sharing research among colleagues, referring to it as ‘mini-building blocks which 
make you the teacher you are’. 

A primary headteacher posed an important question: she recognised that the ‘lay- 
ers of management have a responsibility’ to disseminate research information, 
but queried ‘who does it for managers?’. Later on in the discussion, the same 
headteacher suggested that LEA advisers are ‘the people whose job it is to know 
what is current and where to point you’. 

LEAs have the potential to provide mediation and networking roles. The 
research team explored this with four LEA advisers, asking about their role in 
relation to sharing information about educational research. All four intervie- 
wees were from authorities which were known to have an interest in promoting 
research engagement. 

The interviewees were all able to identify key networks in their authority where 
research did or could potentially play a part. These included: 

• contributing to Continuing Professional Development (CPD) initiatives, such 
as Professional Learning Academies and training programmes for school staff 
(including Advanced Skills Teachers, trainee teachers, newly qualified teach- 
ers and teaching assistants) 

• discussion within school networks (e.g. primary schools, networked learning 
communities) 

• contributing to a specific encouragement of practitioner research (e.g. through 
LEA networks) 

• forming part of accredited postgraduate courses (some of which had been 
developed in partnership with the LEA) 

• forming part of the information provided to schools (e.g. through intranet sites 
and newsletters) 
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• informing advisers’ own responsibility for broad areas of work, such as school 
improvement or remodelling 

• helping to meet the information and CPD needs of advisory teams. 

The advisers acknowledged their own role as leaders and mediators. All four said 
that their role included sharing information (including research information) with 
others. For example, one described herself as: ‘acting as a facilitator and giving 
pointers to teachers’. One went further, stating that his role included: ‘valuing and 
giving permission to research. It is still a matter of addressing hearts and minds, 
raising the status of research and giving messages affirming its importance.’ 

In order to provide a mediating role for research information, LEA advisers 
themselves need to be well informed. The four LEA interviewees said they 
accessed educational research in a variety of different ways, most commonly 
through professional journals and the internet. Two LEAs used a document sum- 
mary service, which provided brief synopses of key educational publications, 
such as official reports and strategy documents. However, it could be difficult for 
them to lit this responsibility into a busy schedule. One adviser said: ‘Printed 
materials are not used in a structured way. For example, I have on my desk an 
article on mathematics teaching which someone has copied for me.’ 

4.4 Accessing research through continuing 
professional development 

Both the LEA advisers and the focus group participants were asked about the role 
of research in CPD. 

The four LEAs had differing organisational approaches to CPD. For example, 
whereas one adviser had direct responsibility for organising training courses, 
another LEA had delegated CPD to an independent provider. One adviser had 
overall responsibility for professional development of staff at an early stage in 
their careers. She mentioned that her authority had just made a commitment that 
all their courses would be ‘supported by up-to-the-minute research’. 

Focus group participants also mentioned the role of research information within 
CPD. Three areas of continuing professional development were discussed by the 
focus group practitioners: 
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• conferences, seminars and workshops 

• formal programmes of study such as foundation courses, masters degrees and 
doctorates 

• research projects with LEA adviser and/or academic support. 

Seminars, workshops and courses were acknowledged to be important sources of 
research information, as one practitioner said: ‘When you go on courses, they 
tend to summarise research for you.' There was general agreement that attending 
a training event where a particular piece of research was ‘well presented’ made 
an enormous difference to the ways in which practitioners engaged with research 
and how they reflected on their teaching. 

In particular, practitioners mentioned speakers on the subjects of assessment for 
learning and ‘brain-based’ learning who presented their own classroom-based 
research in an engaging, confident and inspirational way. National conferences of 
headteachers and, in one case, the International Conference of Principals, were 
seen as important fora for the dissemination of evidence-based practice. 

Several participants talked about the impact of hearing research presented by an 
inspiring researcher. A headteacher of a primary school said she was ‘inspired by 
people’ - a comment that was echoed by other practitioners in other focus 
groups. She recalled presentations by two researchers (Louise Stoll and Dylan 
Wiliam). These sessions had prompted her to search out recommended research 
publications. Similarly, a primary teacher, commented that ‘listening to Alistair 
Smith [a researcher] profoundly changed the way I taught and 1 did go off and 
read the books’. 

Involvement in formal programmes of study provided opportunities for engage- 
ment with research and theory. 



A teaching assistant in a primary school explained that studying for a founda- 
tion degree had helped her to open her mind to educational theory. In her 
opinion, those who were teaching in the 1980s did not seem to value educa- 
tional theory. She had read work by Tina Bruce (an early years researcher) and 
realised that she had ignored Piaget and Vygotsky in the past. Theory seemed 
more relevant now and the course had made her read more. 
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A teacher in the same school agreed that theories are becoming more impor- 
tant for teachers. She was taking part in a course on thinking skills, which 
made reference to Piaget and Vygotsky. She said, ‘In my PGCE [postgraduate 
certificate in education], there was no reference to these theories, but I have 
found them useful.’ 



Practitioners engaging in further study said this had made an impact on how they 
read research articles. For these people, a good set of references was important to 
enable follow-up reading and cross-referencing between research publications. 

Several of the focus group discussions touched on notions of professionalism. 
For example, a primary school headteacher suggested that there had been a 
change in culture recently, resulting in a tide of critical thinking in schools: ‘Now 
there is a heavier emphasis about how research informs practice in school, teach- 
ers are more into questioning what they are doing.’ 

Some FEAs had developed an authority-wide policy of encouraging school staff 
to engage with research. They supported schools in developing action research 
projects alongside national initiatives such as the Best Practice Research Schol- 
arships and with support from leading academics such as John McBeath. 
Practitioners acknowledged an important link between involvement in their own 
research and interest in the findings of national research studies. A head of 
humanities talked about the role of research in professionalism: 

It’s part of the professionalism of the job - before it was about being a teacher 
teaching in the classroom, it’s definitely changing... Research is part of the 
professionalism that we need to reclaim for ourselves, like doctors and 
lawyers have. We are now finding out about the pedagogy of teaching and the 
craft of it. 



4.5 Key messages on practitioners’ access to 
research 

• Focus group participants see the internet as the first point of call for finding 
relevant research. 

• They also access research findings through printed journals and newsletters. 
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• Mediators have a central role to play in disseminating research throughout 
the education sector. Key mediators included LEA staff, trainers, headteach- 
ers/principals and other staff with posts of responsibility. 

• Mediators need access to concise and relevant information that is easy to pass 
on to others. 

• CPD provides opportunities for practitioners to access and become interested 
in research. Inspirational speakers are important catalysts for engaging practi- 
tioners with research. 

• The climate of research engagement (propagated by LEAs and school leaders 
amongst others) plays an important part in whether and how practitioners 
interact with research. 
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5 Responses to the Bulletin 



This section contains information from both the LEA advisers and the focus 
group participants concerning their responses to the NERF Bulletin. The second 
edition of the Bulletin was sent to participants in advance, although not all partic- 
ipants had had the chance to study it in detail before the interviews took place. 

The four LEA advisers had a range of responses to the Bulletin. One adviser was 
already aware of it and using it in his practice. He said: 

It [the Bulletin] informs my thinking, sharpens my focus and keeps me rele- 
vant. I send parts round to other colleagues ( such as members of the local 
authority senior management team and lead senior advisers). I take digest- 
ed elements of it to put in the local authority paper and on the intranet to all 
schools. 

The other three advisers were not aware of the Bulletin until they were 
approached by the NFER team. One of them was not well disposed towards it, 
finding it ‘difficult to penetrate’ and being unsure of its purpose. Not surprising- 
ly, this adviser was not committed to using it in future. The other two were more 
positive. They suggested that they could recommend articles to colleagues, put 
the Bulletin on websites/intranet sites and use it in CPD. One said: ‘I am in the 
process of setting up a CPD website for teachers and a link to the Bulletin could 
go on this. 1 also intend to raise awareness about it at conferences.’ The other had 
decided to recommend one of the articles (on behaviour management) to the 
mentor for graduate teachers. 

Only one of the focus group participants had seen the Bulletin prior to being sent 
a copy for the purpose of the evaluation. Nevertheless, the idea of the Bulletin 
was broadly welcomed. 

One practitioner said: ‘Looking at the NERF Bulletin, I can see that there are a 
lot of things which could benefit my research.’ 

The positive responses from focus group participants can be grouped into three 
broad categories. There were statements about the relevance of the content and 
the level of detail, the opportunity for readers to find out more about a specific 
article, and the relative ease with which readers could navigate the Bulletin. 



First, the clear relevance of articles to practitioners’ work was noted by partici- 
pants in all settings. The substantial, and sometimes demanding, nature of the 
articles was also appreciated and, for many participants, was seen as preferable to 
more attractively presented articles which were perceived as slight or lacking in 
gravitas. The comment from one practitioner, 4 You’d probably learn more from 
[The] NERF [bulletin] because the research is more detailed’, was typical. 

Second, there were positive comments about the use of design features to enhance 
the readability of the Bulletin. In particular, the participants liked the use of: 

• summaries (‘because you can see what is relevant’, ‘it’s a summary that draws 
you in’) 

• text boxes (which allow you to ‘jump in and out’, ‘dip into it’ and ‘stand out 
when speed reading’) 

• clear headings 

• jargon busters (‘saves you looking up words’) 

• a front cover with (‘short, sharp keywords’). 

As a result, it was felt that the Bulletin was ‘easy to find your way round’ . 

Third, the participants appreciated the opportunity presented to them, via the ‘hot 
websites’ and ‘how do we know this?’ sections, to follow up an article by finding 
more information. This was seen as a key advantage of the Bulletin compared 
with other publications. 



One primary teacher commented on the value of using the Bulletin as a means 
of celebrating success. She made the point that, in some cases, the research in 
the Belletin, although recent and potentially challenging to established prac- 
tice, would be familiar to practitioners. This could have the effect of raising 
morale since it would signal that practitioners were in the vanguard of current 
educational thinking. She commented that ‘when reading the Bulletin, it was 
interesting to see what we were doing already’ and made reference to the arti- 
cle on Assessment for Learning: ‘we’re quite hot on this here’. 



However, there were a number of negative comments about the Bulletin, centred 
on its unattractive appearance. This was thought to counteract attempts to make 
the Bulletin readable and easy to navigate. Comments concerned the ‘dense’ 
appearance of the page design and layout, and the lack of a separate front cover. 
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Practitioners in the early years and primary groups were particularly emphatic that 
they disliked the appearance of the Bulletin, due to its lack of colour and pictures. 

The following comments are typical of those made by participants in relation to 
the appearance of the Bulletin: 

The front page is more like the contents page. It needs a front cover. 

I might read the Bulletin, although at the momen t the front cover and colours 
used wouldn ’t really tempt me to pick it up if it was lying around. 

If you want people other than leaders to read something like this, it needs to be 
shorter. 

There’s no ‘scribble space’ to write notes. It’s too packed. 

I would rather see one article on one page. 

There are different things on the same page so it’s not easy to read. 

One participant from a sixth form college commented on a perceived lack of 
rationale to some aspects of content: 

I found some things quite odd - like ‘How do we know this?’ - sounds like 
something you would find in a primary school book. Something I found quite 
difficult was the definition of effect size which left me none the wiser. However, 
other things were expanded which didn 't really need to be. 

She considered this to be indicative of a lack of rationale that contributed to an 
‘uneven tone’ among the articles. A teacher at the early years centre made a simi- 
lar point, wondering what market the Bulletin was aiming for: ‘It’s funny to have 
some articles with basic information and then others with more complex articles 
in a magazine like this. Was it [for] NQTs or more experienced teachers?’ 

Some participants questioned the concept of a product aiming to be relevant to all 
sectors of education and all subjects of the curriculum. For example, practition- 
ers in the early years centre felt that the Bulletin lacked articles with relevance 
for their sector. A secondary headteacher said that he would have to photocopy 
each part of the Bulletin separately to distribute to different subject specialists. In 
his opinion, a themed publication would be preferable: ‘a single issue bulletin is 
much easier for schools to manage.’ This comment was echoed by one of the 
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LEA advisers, who described the Bulletin as ‘a scattergun approach’. This made 
it difficult to decide what to do with the information. In his view, it would have 
been much easier to use if the research had been thematic, because then he could 
have passed it onto a relevant colleague . 

From discussion in all focus groups, it was clear that, because the participants 
could not rely upon all the Bulletin’s content to be relevant to their particular 
context, they would only choose to read and use parts of it, as one practitioner 
said: ‘I would be tempted to skim-read it, but I don’t think I’d be tempted to read 
the whole thing.’ 

A practitioner from an FE college summed up the views of many of the focus 
group participants who had made the effort to engage with the Bulletin despite its 
unattractive appearance: ‘it’s actually much more interesting than it looks.’ 

5.1 Key messages on responses to the Bulletin 

Positive responses to the Bulletin included the following points: 

• it was useful for informing practitioners’ thinking 

• it was relevant to practitioners’ own classroom practice and/or research 

• it was seen to contain credible evidence 

• certain design features enhanced its readability 

• it referred to sources of further information. 

Negative responses to the Bulletin centred on one point: 

• the multiplicity of sectors and themes made the Bulletin’s content difficult to 
access. 
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6 Considering the Bulletin in relation 
to other products 



Practitioners were asked to look at two other printed publications and comment 
on them in comparison to the Bulletin. Publication A is a journal that is produced 
twice a year and aims to provide practical applications of research for headteach- 
ers and teachers. Publication B is a monthly glossy magazine in an A5 format 
that is aimed at school leaders. It contains ‘news’ items and features policy rather 
than research, although it may include reference to research findings. Focus 
group participants had not necessarily seen these publications before. Their com- 
ments therefore tended to focus on surface-level features such as the design and 
layout, rather than an in-depth appraisal of content. 

Practitioners in all sectors commented that they liked different aspects of the 
three publications. They liked the concept and content of the Bulletin, but pre- 
ferred the appearance and layout of publications A and B , which made better use 
of space, colour and photographs. For example, staff at a primary school liked 
different aspects from all three publications. They wanted the A5 size and layout 
of publication B , but wanted to retain some of the ideas in the Bulletin such as the 
‘jargon busters’ and website addresses. 

Much of the discussion centred on the need to match reliable research evidence 
with attractiveness and ease of navigation. A practitioner in the early years centre 
said: ‘Publication A is easier to read because it’s well set out, but you’d probably 
learn more from the Bulletin because the research is more detailed - precise and 
concise’. She explained that her reason for saying this was that the Bulletin con- 
tained more quantitative data. A practitioner at a secondary school said: 

The one that has the most rigour [the Bulletin], is the hardest to read and the 
one with the least rigour [publication B] is the most appealing to read. You 
need people to pick it up, but to have enough rigour for them to go back to it 
another time. 

When comparing the three publications with one another, practitioners made sev- 
eral interesting comments about the concept, design and content of publications 
aimed at a practitioner audience. 



6.1 Concept and purpose of different publications 



Practitioners recognised that the three publications had different (and comple- 
mentary) purposes. They thought that the Bulletin compiled short snippets about 
different research reports so that teachers can go on to read further research if 
they wished to. On the other hand, publication A contained longer, more compre- 
hensive research articles. Publication B was seen as ‘a magazine’, which was 
‘news led’ rather than containing detailed research. One practitioner from a sec- 
ondary school remarked ‘there’s no depth to it’ [publication B] and ‘it leaves you 
hanging’. 

Another secondary school participant explained, ‘the Bulletin has been put 
together for teachers and you can flick through several articles, whereas for [pub- 
lication A] you are more likely to read one or two articles’. 

Practitioners recognised that the content of the publications differed depending 
on their purpose. They felt that the Bulletin was a particularly useful addition for 
busy practitioners because it enabled the reader to get a quick overview of 
research findings which they could follow up if they wished to by further reading 
or accessing the recommended websites. 

6.2 Comments on differences in design and layout 

As noted above, practitioners commented that they preferred the design and lay- 
out of publications A and B over the Bulletin because they were clearer, more 
attractive and easier to skim read. For example, one secondary teacher comment- 
ed that ‘the diagrammatic information is a good idea [in publication A]’. 

Practitioners differed in their liking of the ‘glossy magazine’ look of publication 
B . Several said it would ‘draw them in’ and that they would read it from ‘cover to 
cover’, but others said it did not appeal and was too ‘overdone’. One practitioner 
from a secondary school commented, ‘I would question the validity of the con- 
tent because it’s so glossy.’ On the other hand, several practitioners liked the A5 
format of publication B because it would be easy to carry around and read whilst 
travelling. 

Design features of the three publications which practitioners found particularly 
attractive were: 
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• clear layout, using signposting, bullet points and text boxes 

• use of case studies 

• an attractive front cover indicating the content of the publication 

• use of visual elements and colour 

• comprehensive references and sources of further information. 

6.3 Websites containing research for teachers 

As mentioned above, practitioners said they commonly used the internet to find 
research. The NFER team showed focus group participants examples of research 
pages from two UK websites. The majority of practitioners preferred features 
from one of the two. Listed below is a summary of practitioners’ comments about 
what they particularly liked about website A. 

• It clearly highlighted the relevance and practical implications of the research. 
One secondary school practitioner said she would go straight to the section 
titled ‘what are the implications for staff, parents and carers?’. 

• It had accessible language: two primary school teachers agreed that it was 
‘very readable and user friendly’. 

• It had a limited amount of text on one page. Some practitioners commented 
that although there was the same amount of information on both websites, on 
website A the content had been cleverly laid out on several different pages 
with a link down the left hand side to each new page. As one teaching assistant 
explained, ‘it gives you a little taster, it makes you want to click on it.’ 

• It had clear, bold headings and signposting (e.g. the use of bullet points). 
Teachers stressed the importance of scanning over websites to find relevant 
information in a limited time. 

• It featured key messages on the first page. As one practitioner in a sixth form 
college commented: ‘I like the site as it gives the salient points quickly.’ 

• It had tools such as ‘printer friendly options’ and ‘email to a friend’ , the search 
engine and keywords. Participants explained it was very important to be able 
to print material from websites so they could read it at their leisure and make 
notes on the downloaded pages. 
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There were two features of website B which practitioners appreciated: 

• a clear site map on the left hand side to allow for easy navigation. 

• the use of case studies. 

Although website A was well liked, there were two aspects of it which practition- 
ers disliked. First, the web address at the bottom of page did not print out 
properly. Practitioners felt it was very important that this printed out, so that they 
could return to the source if they needed to. Second, it had a long and complicat- 
ed title. As one practitioner in a primary school commented, ‘I want something in 
teacher language.’ 

6.4 Key messages from a comparison between 
research-based products for practitioners 

Practitioners recognised that different publications served different purposes. 
Nevertheless, they liked printed publications which had: 

• relevant research content coupled with practical applications 

• accessible language 

• case studies of practice 

• an attractive front cover 

• clear layout and good signposting 

• use of colour and illustrations 

• comprehensive references and sources of further information. 

The above points were reiterated for web-based products. In addition, practition- 
ers liked research on websites to have: 

• a limited amount of text on each screen 

• clear sections with easy navigation 

• good printing capabilities 

• tools such as search facilities and ‘email to a friend’. 
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7 Conclusions and recommendations 



Phase two of this evaluation has served to reinforce several of the messages from 
previous research. It has also provided a considerable amount of detail on the 
features of research products that are more or less appealing to practitioners. 

Our main conclusion is that the Bulletin can make an important contribution in 
communicating research to teachers. It will add to and complement existing 
products. The participants in this study saw the Bulletin as contributing to an 
extended view of professionalism, whereby teachers and other staff become 
more reflective, better informed and willing to change their practice to achieve 
better outcomes for their pupils. 

Practitioners recognised that the content of publications differed depending on 
their purpose. They felt that the Bulletin was a particularly useful addition for 
busy practitioners because it enabled the reader to get a quick overview of 
research findings which they could follow up if they wished to by using the rec- 
ommended further reading or accessing the recommended websites. 

One of the consistent themes to emerge was that in order to be useful, research 
must be relevant to practice. But it is important to recognise that relevance is a 
multifaceted concept. Practitioners wanted research to be relevant to their phase 
and to demonstrate an empathy with classroom practice. Research into topical 
issues was considered to be highly relevant, as was educational theory. However, 
relevance also varied according to personal preferences, career pathways, 
involvement in further study and engagement in research. Participants also paid 
tribute to the importance of personal contact with inspiring researchers or with 
colleagues who recommended research to them. 

As well as the importance of relevance, this study has reinforced the multifaceted 
nature of knowledge transmission. Teachers may encounter research through 
magazines, newsletters and the press. Research does little more than raise aware- 
ness among some, but for others it may lead to further reading, reflection and 
even impact on their practice. Research may reach teachers directly, or indirectly 
though influencing policy-makers, opinion leaders and trainers. 



7.1 Recommendations 



The recommendations arising from this study are simple and straightforward. 

• NERF should continue to produce the Bulletin. 

• The Bulletin should be redesigned to make it more visually appealing. 

• Thought should be given to increasing the relevance of the Bulletin for differ- 
ent phases of education. Grouping items together in relation to educational 
phases and/or themes would help to increase its appeal. 

• The Bulletin should be promoted to teachers using both direct and indirect 
pathways (e.g. through key publications, websites, organisations and roles). 
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Appendix 1 Sample description and 

methodology 



Phase two of the evaluation included a small number of focus group interviews 
and telephone interviews. The evaluation took place in the spring of 2005. 

Focus groups were conducted in seven institutions across four educational phas- 
es (nursery, primary, secondary and FE). One focus group was carried out in an 
early years centre and two in each of the other sectors. The institutions differed 
considerably in size, from a primary school comprising approximately 200 
pupils, to an FE college with approximately 12,000 part- and full-time students. 
Each focus group included a broad range of practitioners, such as headteachers, 
teachers/lecturers, nursery nurses and teaching assistants. In the FE college, staff 
predominantly came from the information and communications technology and 
business department. The groups ranged in size from five to eight individuals (a 
total of 40 practitioners) . 

Institutions were identified from: 

• those who were involved in the NFER research project on investigating the 
research-engaged school 

• through personal contacts from previous work carried out both at NFER and 
other institutions 

• recommendations from other colleagues. 

The institutions that were willing to take part were particularly interested in find- 
ing out more about research in general and about the Bulletin. Therefore, it must 
be noted that the sample of institutions we visited was likely to be atypical of 
schools and colleges across England. Every institution we visited was positive 
about reading and using research but each was at a different stage of becoming a 
research-engaged community. In the FE institutions, secondary schools and one 
primary school, practitioners were conducting their own research, both in terms of 
small and large-scale projects. Some practitioners had either gained or were study- 
ing for further qualifications, particularly those in the secondary and FE sectors. 

A schedule was developed that focused on a range of issues, including: 

• research that practitioners had used in their work 



• the types of research they relate to 

• the methods they use to find out about research 

• their views on the Bulletin and other publications and websites. 

The group sessions lasted an hour and took place on the school premises at a con- 
venient time for participants. The sessions were recorded with the permission of 
the participants and the notes were transcribed. 

During spring 2005 , telephone interviews were conducted with four LEA officers 
with responsibility for coordinating the CPD of teachers and lecturers. The selec- 
tion of interviewees was based on personal contacts. 

The purpose of the semi-structured interviews with advisers was to find out about 
how research informed teaching practice in their authority and how the Bulletin 
may contribute. More specifically we asked them about how they kept abreast of 
research, how research is presented to teachers or lecturers and their thoughts on 
how they may use the Bulletin. Each interview lasted approximately half an hour. 



appendix 1 



25 



References 



BLACK, P. and WILIAM, D. (1998). Inside the Black Box: Raising Standards 
Through Classroom Assessment. London: King’s College, London. 

BLACK, P„ HARRISON, C., LEE, C., MARSHALL, B. and WILIAM, D. 
(2002). Working Inside the Black Box: Assessment for Learning in the Class- 
room. London: King’s College, London. 

HARGREAVES, D.H. (1996). Teaching as a Research-based Profession: Possi- 
bilities and Prospects (Teacher Training Agency Annual Lectures 1996). 
London: Teacher Training Agency. 

HEMSLEY-BROWN, J. and SHARP, C. (2003). ‘The use of research to improve 
professional practice: a systematic review of the literature’, Oxford Review of 
Education, 29, 4, 449-70. 

MacBEATH, J„ BOYD, B„ RAND, J. and BELL, S. (1996). Schools Speak for 
Themselves: Towards a Framework for Self-Evaluation. London: NUT. 

NATIONAL COLLEGE FOR SCHOOL LEADERSHIP (2003). Leading the 
Research-engaged School: Why and How School Leaders Engage with Educa- 
tional Research [online]. Available: http://newportal.ncsl.org.uk/media/F7B/94/ 
randd-engaged-overview.pdf [13 April, 2005]. 

NATIONAL EDUCATION RESEARCH FORUM (2001). Building Research 
Capacity [sub-group report, chaired by Alan Dyson]. London: NERF. 

RICKINSON, M„ ASPINALL, C., CLARK, A., DAWSON, L„ McLEOD, S., 
POULTON, R, ROGERS, J. and SARGENT, J. (2003). Connecting Research 
and Practice: Education for Sustainable Development . Southwell: BERA. 

TAGGART, G., KENDALL, L„ BENTON, T. and SHARP, C. (2004). Evalua- 
tion of the NERF Bulletin Trial: Phase One Report. Unpublished report. 

WALTER, I., NUTLEY, S. and DAVIES, H. (2004). Assessing Research Impact: 
Final Report of the RURU Seminar, 15-16 January 2004 [online]. Available: 
http://www.st-andrews.ac.uk/%7Eruru/publications.htm [25 April, 2005]. 

WILSON, R„ HEMSLEY-BROWN, J., EASTON, C. and SHARP, C. (2003). 
Using Research for School Improvement: the LEA’s Role (LGA Research Report 
42). Slough: NFER. 



f 4$ 




If you would like to receive a printed 
copy of this report, please contact the 
Publications Unit on 01753 637002 or 
book.sales@nfer.ac.uk 

ISBN 1 903880 73 4 
NFER Ref: EEB 

National Foundation for 
Educational Research 
The Mere, Upton Park 
Slough, Berks SL1 2DQ 
www.nfer.ac.uk 



